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It is by now established that neutrinos mix, have (different) non-zero 
masses, and therefore oscillate. The oscillation parameters themselves, how- 
ever, are not all well-known. An open problem is that of the neutrino mass 
hierarchy. We study the possibility of determining the neutrino mass hierar- 
chy with atmospheric neutrinos using an iron calorimeter detector capable of 
charge identification such as the proposed monolith and ical/ino detectors. 
We find that such detectors are sensitive to the sign of the mass-squared dif- 
ference, (532 = m\ — ml, provided the as-yet unknown mixing angle between 
the first and third generations, ^13, is greater than 6° (sin^ 2^13 > 0.04). A 
result with a significance greater than 90% CL requires large exposures (more 
than 500 kton-years) as well as good energy and angular resolution of the 
detected muons (better than 15%), especially for small 6*13. Hence obtain- 
ing definitive results with such a detector is difficult, unless ^13 turns out to 
be large. In contrast, such detectors can establish a clear oscillation pattern 
in atmospheric neutrinos in about 150 kton-years, therefore determining the 
absolute value of ^32 and sin^ 2023 to within 10%. 
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I. INTRODUCTION 



Recent experimental results from the observations of solar, atmospheric and reactor neu- 
trinos have provided compelling evidence for neutrino oscillations [1-3]. Until recently the 
evidence for neutrino oscillation was indirect, with a small window still available for other 
possibilities such as neutrino decay. The latest results from the Super-Kamiokande (Super- 
K) collaboration [4], presenting the evidence for an "oscillation minimum" in the observation 
of atmospheric neutrinos, seem to provide for the first time clear and unambiguous evidence 
for the oscillation hypothesis [5,6] and hence of non-zero (and different) neutrino masses and 
mixing. Recently, data is also available from the long-baseline experiment K2K [7] . 

Clearly the focus from now on will be to determine precisely the mixing parameters 
and mass squared differences apart from the observation of the oscillation pattern beyond 
the first minimum. An important question that still remains open is the hierarchy of the 
masses. Within the three neutrino mixing framework, the results on solar neutrinos prefer 
the dominant mass eigenstates in to have the hierarchy m2 > mi so that the mass-squared 
difference^ ^21 = m^—rril > 0. The sign of the other mass-squared difference ^32 (equivalently 
S^i since it is so much larger than ^21) is not known. 

The issue of the neutrino mass hierarchy (and measurement of the (13) mixing angle) was 
discussed by Tabarelli de Fatis [8] through a measurement of muon neutrino and anti-neutrino 
events in a magnetised iron calorimeter. In a recent analysis, Palomares-Ruiz and Petcov 
[9] have discussed the resolution of the issue of hierarchy of neutrino masses in magnetised 
iron calorimeters by looking at the ratio (and difference) of events with and //~ due to 
atmospheric muon neutrino and anti-neutrino interactions. 

In principle an iron calorimeter (ical) detector with charge identification capability as 
has been proposed by the monolith [10], MiNOS [11] and ical/ino [12] collaborations, with 
a mass of at least 30-50 ktons, can resolve this issue; the question at hand is one of feasibihty 
and exposure times. 

In this paper we focus on a new difference asymmetry thaX is especially suited to probe the 
mass hierarchy. This asymmetry is constructed from observables involving the detection of 
muon neutrinos (and anti-neutrinos) through charged current interactions with muons in the 
final state. In an earlier work [9] it was shown that the direct and inverted hierarchies could 
be discriminated by studying the magnitude of certain asymmetries; in contrast, the new 
asymmetry probes the hierarchy issue via its sign, which is surely a more robust signature. 
Provided sin^ 26*13 > 0.04, an iCAL detector is sensitive to the mass hierarchy. A result with 
a significance greater than 90% CL requires large exposures of more than 500 kton-years 
as well as good energy and angular resolution of the detected muons (better than 15%), 
especially for small ^13. Hence obtaining definitive results with such a detector is difficult, 
unless ^^13 turns out to be large. 

Furthermore, such an asymmetry is not as sensitive to the magnitude of ^13; in the case 
of a positive result it may only place a lower bound on this parameter. The issue of ^13 



^All oscillation experiments are sensitive to mass-squared differences rather than the absolute scale 
of the masses themselves. 
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therefore may not be resolved in such detectors using atmospheric neutrinos alone. 

In section 2 we present some details of the oscillation probabilities and their dependence 
on masses and mixing parameters after summarising the status of their determination within 
the three neutrino frame-work. Some features of these probabilities within some reasonable 
approximation such as single-scale dominance are presented in a semi-analytic treatment. 
The new asymmetry and its sensitivity to the hierarchy are discussed here as also the crucial 
question of binning the data to maximise this asymmetry. In section 3 we analyse the results 
numerically for the event rates both together and separately for different charged lepton final 
states using the nuance code [13] for atmospheric neutrinos distributed according to the 3-D 
Honda flux [14] . We discuss the sensitivity of the ICAL detector to the hierarchy of neutrino 
masses. This obviously depends on the mixing parameters. We highlight the sensitivity to 
the two unknown parameters, the (13) mixing angle and the sign of ^32. In section 4 we 
generate events using the NUANCE neutrino generator and study the impact of statistical 
errors and fluctuations on the rates and asymmetries. The effect of detector resolution and 
cuts on the observables is also discussed. In section 5 we present our conclusions. 



II. THREE FLAVOUR OSCILLATIONS 

An understanding of the gamut of data from solar, atmospheric and reactor neutrinos 
requires a mixing scheme with at least three neutrinos in which the neutrino flavour states 
\^a)-i (tt = e, /X, r) are linear superpositions of the neutrino mass eigenstates \ui), {i = 1, 2, 3), 
with masses : 

ka) = X^t^mki) • (1) 
i 

Here U is the 3x3 unitary matrix which may be parametrised as [6] (ignoring Majorana 
phases) : 



/ C12C13 S12C13 sise 

~C23Si2 — •S23'5l3Cl2e"^ C23C12 — •§23*13*126"^ •523C13 
V S23S12 - C23Sl3Ci2e*'' -S23C12 - C23Si3Si2e"' C23C13 / 



(2) 



Here C12 = cos ^12, S12 — sin ^12 etc., and 5 denotes the CP violating (Dirac) phase. By 
definition, the 3x3 neutrino mass matrix Mj, is diagonalised in the charged-lepton mass 
basis by U : 

U^M^U ^ dia:g{mi, 1112,1713). (3) 

The parameters involved are the mixing angles 9ij and the mass-squared differences Sij = 
— m|. A combined analysis [15] of the data from all the experiments gives the following 
best fits to the oscillation parameters: 

• A combination of solar neutrino experiments and the KamLAND reactor experiment 
gives 

5.4 X 10"^ eV^ < 621 < 9.4 x 10"^ eV^ , 

0.30 < tan^ Ou < 0.64 , (4) 
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with the best-fit values given by S21 — 6.9 x 10 ^ eV^ and tan^ 612 — 0.40. Note in 
particular that m| > m^. 

• A combination of atmospheric neutrino experiments and the K2K experiment gives 

1.4 X 10"^ eV^ < \632\ < 5.1 x 10"^ eV^ , 

sin^ 2^23 > 0.86 , (5) 

with the best-fit values given by \632\ = 2.0 x 10~^ eV^ and sin^ 2^23 = 1- Note that 
there is as yet no determination of the sign of this mass-squared difference. 

• The CHOOZ bound on the effective mixing angle using the above parametrisation is, 

sin^ 2^13 < 0.249 (99.73% C.L.) (6) 

so that < 14.9°. 

Inclusion of matter effects on the neutrino oscillation probabilities has a complicated de- 
pendence on these parameters; furthermore, the oscillation probability can only be computed 
numerically using a reference model for the variation of earth's density. Before wc present 
the exact numerical results, we first discuss a simplified analytical model in vacuum and 
matter. 

A. Probabilities in Vacuum 

In order to focus on the issues involved and their disentanglement we first discuss the 
vacuum oscillations (actually the muon neutrino survival) probabilities. The survival proba- 
bility of a muon neutrino of energy E (in GeV) after traversing a distance L (in km) is given 

by 

-[2C/,,C/,3]^sin^(H^) , (7) 

where we have neglected the CP violating phase. Note that in vacuum the probabilities are 
the same for both neutrinos and anti-neutrinos. In Fig. 1 we plot the survival probability of 
as a function of L/E for ^32 = 2 x 10~^ eV^ and 6'i3 = 9°, with other parameters fixed 
to their best-fit values. For comparison we have also shown the two generation oscillation 
probability, 

P(j = l-sin^2^23sin^(i^) , (8) 
which has been widely used to fit \Ss2\ and sin ^23 from data. 
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Notice that the three generation formula reduces to the two generation one only after 
setting ^^13 = and 621 = 0. Even though there are two oscillating terms (frequencies) 
controlling the survival probability, the value of the probability at small L/E is dominated 
by the last two terms in eq. 7. In turn, this implies that the first observable probability 
minimum is roughly independent of both ^21 and ^13, even for ^13 as large as the CHOOZ 
limit. At larger values of L/E, when S^iL/E becomes large, the subdominant frequency 
controlled by ^21 has substantial effect. 

This can be seen from Fig. 1 where both the two-flavour and three-flavour probabilities 
are plotted as a function of L/E. In fact, for very large L/E, and for this typical set of 
oscillation parameters, the vacuum probability P^^ averages to ~ 3/8 and not to 1/2 as is 
the case for two generation mixing. 

Notice that the probabilities in vacuum are the same for both neutrinos and anti-neutrinos 
and depend only on the absolute value of the mass-squared differences. However, the inclu- 
sion of three flavours automatically turns on matter effects which alter these vacuum results; 
we now examine the modification of these expressions due to matter effects. 
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B. Probabilities in matter 

In order to simplify the analysis, we first consider the propagation of neutrinos and anti- 
neutrinos produced in the atmosphere in a slab of constant density p (in gms/cc) [16]. In this 
case, we can retain the same expressions as in vacuum for the probabilities, see eq. 7, but 
replace the mass-squared differences and mixing angles by the corresponding matter- modified 
effective values obtained by diagonalising the matter dependent matrix (Hamiltonian): 

(9) 

Voo(53iy voooy 

where 

A = 2V2GFneE = 7.63 x 10"^ eV^ p(gni/cc) E{GeV) eV^. (10) 

Here Gp and Ug are the Fermi constant and electron number density in matter and p is 
the matter density. Further simplification arises because 621 ^ ^31 and we can treat the 
propagation in matter as a one mass-scale problem involving only ^32 ~ ^31. The matter 
dependent mixing angle 9i2,m may be approximately written as 

sin 2^12 , , 

sin2^i2,m ~ / == ■ (11) 

y (cos 2^12 - {A/S21) C0S2 ^^13)2 + sin2 2^i2 

The condition for "resonance" in the (12) channel is given by ^21 cos 2^12 = Acos^ 9is which 
roughly translates to the condition, pE ^ 0.5. We note in passing that the resonance in the 
(12) sector for atmospheric neutrinos has a very different character from its solar counterpart. 
For example, for energies above 1 GeV, the resonance occurs at extremely small densities, 
even in the atmosphere itself for some energies ! Furthermore, the resonance width which 
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is determined by sin^ 9i2 is rather broad, with the tail extending below A = so that the 
effect of matter in this sector on both neutrinos and anti-neutrinos will be roughly similar. 

This also implies that sin 2^i2,m is rather small for neutrinos or anti-ncutrinos of few GeV 
energy even in low density regions such as the earth's mantle — in fact at high densities and 
energies, we have 



sin 26i2,r, 



sin 2^12 
pE 



(12) 



so that one can neglect the term depending on 821 for multi-GeV atmospheric neutrino 
propagation in earth matter. Indeed the same argument also holds for anti-neutrinos since 
the equation above^ holds for them as well. 

The effect of matter on the angle ^13 is non-trivial and is given by 



sin 26*13 , 



sin 2^13 



^(cOs2^13-(^/532))2 + (sin 2^13)^ 



(13) 



The resonance condition in the (13) channel is then given by ^32 cos 2^13 = A with a width 
5 A — (532sin2^^i3. Since sin^2^i3 is small, the resonance is rather sharp. This condition is 
satisfied for range of values 13 < pE < 40 for 1 < S^i/lO'^ < 3 eV^. The average mantle 
density is about 4-5 gm/cc and the resonance occurs for energies greater than 4 GeV. In 
the core, where the density exceeds 9 gm/cc the resonance condition is satisfied with even 
smaller energy. For anti-neutrinos the matter effects come with opposite sign, A — > —A, so 
that resonance occurs with the inverted mass hierarchy ^31 < 0. 

The main observable effect due to propagation in matter comes from the matter depen- 
dent angle ^13 ^ since for all practical purposes we can set 



sin 26'23,m ~ sin 26*23 • 
The approximate survival probability [16] for i/^ can be written as 



(14) 



pm 
MM 



{A, 5s2) 



1 - 



sin^ 2^23 [sin' O^^rn sin' A^^ + cos' Oi^^rn sin' A^^] 
- sin' 2^13,^ sin^ 623 sin' A^^ , 



(15) 



where 



Am 
^21 



Am 
^32 



Am 
^31 



I.27632L 1 

E 2 

1.27^32^ 1 



sin 2^ 



13 



sin 26'i3,„ 
sin 2^13 



1-^ 

CI32 
A' 



E 2 [Sin2^i3,m <^32 

I.27532L r sin 2^13 " 
E [sin2ei3,„ 



(16) 



^This term is often neglected citing the reason that (^21 /(^ai ^ 1- Actually, 52iL/E can be of 
the order of unity for atmospheric neutrinos; it is the smallness of the coefficient of this term that 
allows it to be dropped. 
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Note that for sufficiently large values of A/ 5^2-, all the three scales are of the same order of 
magnitude including which cannot therefore be neglected. 

If we further take sin^ ^13 to be small, which it is, and restrict the analysis to values of A 
away from resonance, then we can expand the ratio 

5 sin 2^13 ^. 2A5 . 

^— ^{5-A) + sm^ , 17 

sm 26'i3,m 6 — A 

where we have used the notation 5 = 8^2- As a result the survival probabilities for t/^ and 
Vn can be simplified further and written, to order sin^ ^13, as, 

P7M^ S) ^ PjS - sin^ ^13 j^T, + (^^) ' (r, sin^[(5 - A)x] + n) 

^.(A<^)«i^iJ-sin^^i3 "5^^! +(5^) {T2sm'[{5 + A)x]+Ts) , (18) 

where x = 1.27 L/E. Here Pji^J = 1 — sin^ 2^23 sin^(a:5) is the 2-generation muon neutrino 

survival probability that is independent of matter effects as well as sin^^i3. The matter 
dependence appears in the remaining terms, where 



Ti = xS sin^ 2^23 sin 2Sx , 

T2 = 4sin^^23 , 

= sin^ 26*23 (sin^ Ax - sin^ Sx^ 



Notice that the survival probability for the anti-neutrinos, P^^, is given by the replacement 
A — > —A in P^. Both probabilities are invariant under {A, 5) <-> {—A, —5). 
Several important points emerge from this analysis: 

• As was noticed in the analysis of the vacuum probabilities with three flavours, there 
are two mass scales and their associated frequencies. Even though one of the scales 
is small in magnitude, ^21 ^ ^31, it does make an impact on the survival probability 
due to the large L/E available; this happens because a large range of L/E is available 
for atmospheric neutrinos. This makes the 3-flavour results differ strongly from the 
2-flavour ones in the large L/E region, in the vacuum case. 

• The matter effect however effectively removes one of these frequencies controlled by 
821 such that the oscillations arc controlled by the frequency determined by ^32 and to 
a lesser extent by the matter term A. In other words, the deviation of the 3-flavour 
probability from the 2-flavour one at large L/E disappears when matter effects are 
turned on. The residual matter effect in atmospheric neutrinos is thus a sub-leading 
effect. 

• Thus the matter dependent and resemble the two-generation probabiUty curve 
shown in Fig. 1 rather than the three generation one. In fact the first dip is almost 
completely unaltered by the matter effects, and the average behaviour for large L/E 
is similar to the two-generation case. This in fact justifies the attempts to get bounds 
on 1^321 and sin 26*23 using fits to the Super-K data within the two-fiavour oscillation 
formalism. 
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• Therefore the sensitivity to ^13 and the sign of ^32 are hidden in the sub-leading effects 
which show up in the oscillation frequency, the width and the amplitude after the 
first dip, and are maximal for intermediate values of L/E. The matter effect is more 
pronounced for (anti-)neutrinos in the case of the (inverted) direct hierarchy, since the 
amplitude of the matter-dependent term is enhanced (decreased) when ^32 and A have 
the same (opposite) sign, as can be seen from eq. 18. 

All of the above statements apply appropriately (by interchanging the matter effects for 
neutrinos and anti-neutrinos) for both the direct and inverted hierarchies. This exchange 
symmetry will however be broken once the cross-sections are taken into account since the 
neutrino cross-sections are on the average a factor of two more than the anti-neutrino cross- 
sections. Since matter effects are enhanced for neutrinos in the case of the direct hierarchy, 
such a hierarchy will be more favourable for detection due to the larger events sample for 
the same exposure time. 



III. EVENT RATES AND ASYMMETRIES 

The arguments of the previous section clearly indicate that the main effect of matter on 
the survival probability is to reduce it to a single frequency (which however is not a constant 
since it depends both on energy and the density) oscillation, whereas in vacuum there are 
clearly two well-separated oscillations. 

The inclusion of matter effects through a 3-flavour analysis is complicated by an addi- 
tional energy dependence. While the survival probability in the 2-fiavour case depends only 
on the ratio x = L/E, that in 3-fiavours depends explicitly on both x and E, the latter 
through the matter term. The experimental observable is the event rate in the detector. 
The fully differential event rate for neutrinos of flavour a to be detected is given by the 
general expression: 

dA/"" 

^^^^ = KyY.Pp^{E,x) ^p{E,x) aa{E) , (19) 

where x = L/E and is the total interaction cross-section for the a type neutrino to 
interact with the detector material. Here Ppa is the conversion probability of a neutrino of 
flavour /3 to a flavour a. 

The flux-dependent term $c«(£', x) is obtained from the doubly-differential neutrino (or 
anti- neutrino) flux of flavour a, d'^(f)a{E,z)/dl-aEdz, which is a function of the energy E 
and zenith (actually nadir) angle z = cos 9, by multiplying with a suitable Jacobean factor. 

The factor Ky is the detector dependent factor measured in units of kton-years. The 
detector is assumed to be mainly made up of magnetised iron with active detector elements. 
In the MONOLITH and ical/ino proposals, the active detector elements are glass resistive 
plate gas filled chambers (RPCs). In either of these proposals the detector mass is almost 
entirely (> 98%) due to its iron content. We will be interested here in event ratios; hence the 
factor Ky and other actual detector details arc not necessary for the analysis in this section. 
However, we assume that the detector is capable of identifying the charge of the muons in 
the final state with good precision. 
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The distance of propagation L of the neutrino from the point of production to the detector 
is obtained from 



{R + LoY -{RsinOy -Rcose , (20) 

where = corresponds to neutrinos reaching the detector verticaUy downwards after a dis- 
tance Lq which is the average height above the surface of the earth at which the atmospheric 
neutrinos are produced. We take this to be about 15 kms, as is the convention. Here R is 
the radius of the earth. 

The event rate in a given bin ol x — L/E is, 



^ ~ kin ^ Je^,„ E dlnEdx ' ^ ' 



henceforth we discuss only the case of muon-neutrinos, a = fi. For the case of atmospheric 
neutrinos of interest here, both Pe/i and P^^ contribute in the expression above. However, 
due to the smallness of the (13) mixing angle, ^13, the contribution of Pe^ is very small and 
is maximum near large L/E where it is about 10-20% (5%) for neutrinos (anti-neutrinos). 

The event rate is expressed as a function of a;, averaged over a bin width that will be 
appropriately chosen to maximise the sensitivity to the sign of ^32. The expression given in 
eq. 21 is the best case scenario since the integration is over the neutrino or anti-neutrino 
energy. We discuss the effect of including the detector resolution functions in the next 
section. 

In order to proceed further we also distinguish between the up-coming and down-going 
neutrinos (and anti-neutrinos). It is obvious that the down-going particles travel a much 
smaller path-length of 15 < L < 400 km in matter. Hence effects of oscillation are expected 
to be neghgible for these neutrinos^. 

A useful measure of oscillations is the ratio of up-coming to down-going neutrinos with 
nadir/zenith angles interchanged. This is clear from Fig. 2. The fluxes of atmospheric 
neutrinos from directions 6 and (tt— 6*) are expected to be similar in the absence of oscillations, 
especially for larger energies, E > few GeV. Since the path-length traversed, L, is related to 
9 as 

L = /(|cose|) -Pcos^ , 

(see eq. 20), the replacement 9 {n — 9) effectively changes the sign of the second term in 
the equation above, thus taking, for instance, a down-going neutrino to an up-coming one. 
The ratio of events in the up-down directions for a given x = L/E, therefore, reflects the 
asymmetry of the up-down fluxes, due to oscillations. We define [20] 

U, ^ No. of events from up-coming muon neutrinos (x) 



D No. of events from down- going muon neutrinos (x) 



^This is not true for very horizontal neutrinos; however, if the detector geometry consists of 
horizontally stacked iron-platcs, such very horizontal events cannot be observed. 
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where x = x{9 ^ {n — 9)) and the number of up-coming (U) or down-going (D) events is 
calculated using eq. 21. Similarly U/D defines the corresponding up-down ratio for anti- 
neutrinos. Since the effect of oscillations on the denominator is small, the ratio TZ is effectively 
the ratio of oscillated to unoscillated events with the same L/E. 

Thus, while reflecting the effect of oscillations, this ratio also minimises errors due to the 
uncertainties in the overall flux normalisation (which can be as large as 30%). Such a ratio, 
and its sensitivity to the parameters of neutrino oscillations, has been well-studied before; in 
particular, detailed studies have been carried out by the MONOLITH collaboration [10]. We 
shall not repeat the results of such a study here; however, we will use the information on 
the absolute value of ^32 that such a study will yield in order to analyse matter effects and 
its implication for the neutrino mass hierarchy. 

We now proceed with the numerical details. All calculations presented are done using 
the NUANCE neutrino generator. The atmospheric neutrino fiux given by Honda ct al. [14] 
is used. The code generates oscillation probabilities based on the three neutrino oscillation 
framework of Barger et al. [17]. The earth matter effects are calculated using the Prehminary 
Reference Earth Model (PREM) [18] for the variation in matter density. 

Apart from L and E, the probabilities depend on the parameters ^21) ^32, ^12? ^23, and 
^13 (the CP phase has been set to zero). Since the focus of this paper is the issue of matter 
effects and mass hierarchy, we fix the other parameters at their best-fit values'': 

521 = 7.0 X 10-^eV^ ; tem^ 612 = 0.39 ; sin2 2^23 = 1-0 , 

and vary only the parameters of interest, viz., ^32 (both in magnitude and in sign) and ^13. 
We restrict ourselves to the latest Super-K limits on the magnitude of ^32: 1 < [fe/lO"^! < 3 
eV^ and the CHOOZ bound sin2 2^i3 < 0.249(^i3 < 14.9°). 

The NUANCE neutrino generator calculates the probabilities, fiuxes and cross-sections 
suitably in bins of E and cos^ for use in generating events. Event rates, as given in eq. 19, 
were first computed using these tables, with suitable interpolation and integration over the 
variables with a cut on the energy, E'min = 4 GeV. This cut was optimised to maximise the 
matter-dependent effects, as we shall see below. 

We show the variation of the up/down events ratio 7^ as a function of L/E for both z/^ 
and 77^ in Figs. 3 and 4 for direct (m| > mg) and inverted (m^ < m^) hierarchies. Results 
are shown for \632\ = 1, 2, 3 x lO'^ eV^ and for ^13 = 5°(Fig. 3) and ir(Fig. 4) to show the 
sensitivity to these two parameters. 

It is obvious from Figs. 3 and 4 that there is very little sensitivity to the sign of ^32 at 
small values of ^^13. For example, it is clear that atmospheric neutrino experiments can say 
very little about the hierarchy problem below ^13 = 6° (sin^ 2^13 = 0.04). However, the 
sensitivity to the sign increases with the magnitude of 6*13. 

It is important to note that the position of the first minimum is independent of both ^13 
and the sign of ^32 but depends only on the magnitude of the latter. Hence it can be reliably 



^These values have altered somewhat with the latest results from Super-K and KamLAND; how- 
ever, they should not affect our conclusions or even our numerical calculations significantly. 
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used to determine the magnitude of ^32 as a precursor to determining its sign with the same 
experimental set-up. 

It is clear that the neutrino and anti-neutrino up/down event ratios are different from 
each other as well as different with direct and inverted mass hierarchies (due to matter 
effects); the distinction (and hence measurement possibilities) between the two hierarchies 
can be amplified by defining the asymmetry, 

Am{x) ^ ^{x) - ^{x) . (22) 

The asymmetry, calculated numerically, and integrated over £"111111 > 4 GeV is plotted as a 
function of L/E in Fig. 5 for ^32 = 1, 2, 3 x 10~^ eV^. The thick (blue) curves in the figure 
correspond to the direct mass hierarchy (labelled D) and the thin (red) curves to the inverted 
mass hierarchy (labelled I). The curves in each envelope correspond to ^13 = 5, 7, 9, 11 degrees 
(sin^2^^i3 from 0.03-0.14) with the asymmetry increasing symmetrically with ^13 about the 
An = line for direct and inverse hierarchies. It is seen that the direct and inverted 
asymmetries are exactly out of phase. 

The maximum divergence between the direct and inverted hierarchies is smaller in the 
first envelope than in the second; these correspond to the first dip and rise in the up/down 
events ratio (equivalently, in -P^^). We will see in the next section that the statistically 
significant region with maximum number of events corresponds to these first few envelopes, 
i.e., to the range 500 < L/E < 1500 for all the three values of the ^32 considered here. 

For neutrinos of energy about 5-7 GeV the interesting region thus corresponds to dis- 
tances 4000-10000 km with an average around 7000 km [19] which is really the magic baseline 
where matter effects are largest and the effect of the CP violating phase is negligible (though 
we have set this phase to zero). 

In order to understand why the direct and inverted asymmetries are out of phase, it is 
convenient to appeal to the analytic expressions for the probabilities discussed in the previous 
section. 

Since the events in a given L/E bin are largely saturated by the smallest allowed energy 
in that bin (the fiux falls faster than 1 / E"^) , the effect of the cross-section factor is small and 
the up/down events ratio is well- approximated by the flux-averaged probability P^^ deflned 
below: 

r r°° dE 

/ d,r / p^^,(E.x) ^jE.x) 

Pa}m = to d£ • ^23) 

/ dx $,(E,a;) 

Considering only the dominant contribution from P^^ to this expression, we see that 
corresponds to the approximate difference asymmetry, 

Am^A{A,6s2)^PZ-P7,- (24) 

Here P denotes the survival probability for anti-neutrinos which is obtained from the cor- 
responding neutrino survival probability P by the replacement A —A and the explicit 
dependences on the energy and path length E and L have been suppressed for clarity. 
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In the single mass-scale dominance approximation (with ^21 ^31? A), the survival prob- 
ability for neutrinos and anti-neutrinos involves only the ratio A/ 632- As a result we have, 



(25) 



This is easy to see from the approximate expressions for the case of const ant- density matter 
in eq. 15. It follows therefore, for a given E and L, 



for constant matter density. Note that the amplitude is not a constant but depends on the 
energy and distance travelled. In other words, the asymmetry oscillates with L/E while 
being exactly out of phase for the direct and inverse hierarchies. It appears (as can be seen 
from Figs. 3 and 4) that this feature survives in the events ratio computed using the exact 
formula without using the approximation defined in eq. 24 above. 

Note also that the oscillation wavelength is a complicated function of both the matter- 
dependent terms as well as the mass-squared difference, ^32. However, the matter effect is 
not so significant in the case of (anti-) neutrinos with (direct) inverted hierarchy, where the 
oscillation extrema are essentially determined by the matter-independent condition. 



So far, wc have discussed the asymmetry where separation of neutrino and anti-neutrino 
events is a must. This can be arranged through observation of charged- current channels with 
either /i" or in the final state, for instance, through the use of a magnetic field. However, 
in low-counting experiments such as these, it is also useful to ask whether any information 
can be gained from putting the neutrino and anti-neutrino samples together, thus increasing 
statistics. We define 



Note that this does not need charge identification. We show the dependence oi Sn on the 
parameters ^32 (both magnitude and sign) and ^^13 in Fig. 6. We see that the amplitude of 
oscillation depends weakly on 613 and that it dies down faster for the direct hierarchy than 
for the inverted hierarchy. It may therefore be possible to determine the mass hierarchy 
with the sum of neutrino and anti-neutrino data, but for some- what larger values of the (13) 
mixing angle, ^^13 > 7°. 

To sum up, the analysis of the up/ down events ratio, their difference and their sum, 
clearly indicates that 

1. the location of the first dip (for neutrinos and anti-neutrinos with either hierarchy) 
depends only on the magnitude of ^32. 

2. the z/^ — 77^ difference asymmetry is almost exactly out of phase for direct and in- 
verted hierarchies in L/E bins whose widths can be optimally determined by a matter- 
independent condition involving 1^32 1 and L/E. 




(26) 
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3. the amplitude of the asymmetry defined above is clearly a sensitive function of ^13. 

In the next section we examine the event rates and asymmetries in a realistic magnetised 
iron calorimeter detector capable of charge identification such as in the monolith, minos 
and ical/ino proposals. 



IV. EVENT RATES WITH A NEUTRINO GENERATOR 

So far wc have discussed the theoretical up/down events ratios, a difference asymmetry 
and its matter sensitivity. We now need to address the question of statistics, i.e., estimate 
the detector dimension and the time required for data accumulation. We generate events 
using the nuance neutrino generator [13] with the ical/ino geometry which consists of 
140 layers of 6 cm thick iron plates of transverse section 16 m x 32 m. The RPCs are 
sandwiched between the layers. The fiducial mass of the detector is a little under 30 ktons. 
Results shown correspond to 480 kton-years data and indicate (1) the statistical significance 
of the events sample (the error bars) and (2) the impact of fiuctuations (deviation of the 
central values from the predictions of the previous section) . 

The E and L of those neutrinos that undergo charged current interaction in the detector 
to produce muons in the final state were directly used to generate the sums and asymme- 
tries. The effect of finite L/E resolution is discussed in the next section. A detailed study 
reconstructing the neutrino energy and direction from the final state muons and hadrons is 
in progress. 

As stated earlier, it was found that a cut of -Emin = 4 GeV maximises the relative matter- 
dependent effect between neutrinos and anti-neutrinos. While such effects are washed out 
beyond E 10 GeV, we integrated over all possible energies in a bin to improve statistics. 
Data samples were generated for different values of ^13 and ^32 (both signs), keeping the 
other parameters fixed to the values listed in the previous section. 



A. The asymmetry 

To maximise the asymmetry, it is necessary to integrate the events in a bin size that 
includes one half-period of the oscillation, where the asymmetry is always positive or negative 
(see Fig. 5). Unfortunately, this is not easy since different path lengths correspond to different 
matter densities and the wavelength is density dependent. However, recall that the oscillation 
wave-length was roughly constant and matter-independent for (anti-)neutrinos with (direct) 
inverted hierarchy. We therefore identify the bins spanning those L/E corresponding to 

^^ = n-; nodd. (28) 

This has the advantage of being matter-independent. Once 1^32 1 is measured from the 
(matter/hierarchy-independent) first minimum in the plot of the U/D ratio as a function of 
L/E, this can be used to generate the bin sizes for studying the matter-dependent asymme- 
try 
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We exhibit the results for |532| = 1, 2, 3 x 10 ^ eV^ in Figs. 7, 8 and 9 respectively. In 
each figure the events asymmetry is plotted as a function of L/ E. In all zenith 
angle cut of | cos^| > 0.1 was used to remove "horizontal" events, as discussed earlier. The 
first data point corresponds to a region where no matter effect is expected, that is, up to an 
L/E smaller than the n — 1 value in eq. 28 above. 

The "data" points shown correspond to binning the actual events generated using the 
NUANCE Monte Carlo event generator in bins whose widths follow the expression eq. 28. The 
"dots" and the "crosses" correspond to the direct and inverted hierarchy. The theoretical 
asymmetry generated by integrating the differential event rate over the bin width is plotted 
for comparison as a histogram. The darker lines correspond to the direct hierarchy and 
lighter ones to the inverted hierarchy. The deviation of the points from the value indicated 
by the corresponding histogram is an indicator of fluctuations. The size of the error-bar on 
the data reflects only the statistical error due to the data size in that bin. 

The error bars are larger for the inverted than for the direct hierarchy in a given bin. 
This is because it is the anti-neutrino event rate that is sensitive to matter effects in the 
case of inverted hierarchy, as described in section 2. Since the anti-neutrino cross-sections 
are about a factor of two smaller than the neutrino cross-section, the event rates are smaller 
and the error bars correspondingly larger. Statistically speaking, therefore, it is easier to 
detect direct than inverted hierarchy. 

Results are separately shown for ^13 = 5, 7, 9, 11 degrees. It is easily seen that the greatest 
sensitivity to matter effects occurs for a value of 1^32 1 = 2 x 10^^ cV^. For a smaller 1^321, the 
matter effect does not develop signiflcantly until a fairly large L/E, where the event rates 
begin to drop; for larger \5^2\-, the larger number of bins where the asymmetry changes sign 
leads to smaller statistics per bin. 

Note that the first oscillation period (second data point) corresponds to a minimum in 
the up-coming events rate. In all cases, the largest sensitivity to the hierarchy is therefore 
in the region around the first maximum in the up/down ratio, or the third bin. In fact, the 
survival probability is sensitive to matter cflccts for energies roughly in the range 2 < < 10 
GeV. The cut of 4 GeV on the neutrino energy is used to accentuate the asymmetry in the 
third bin. 

A negative (positive) value of asymmetry in this bin clearly indicates direct (inverted) 
mass hierarchy. It may be possible to improve this result by binning over regions of L/E 
taking into account modiflcations due to matter of eq. 28; in fact, with real data, it may 
be also be possible to have variable bin sizes, determined by maximising the magnitude of 
the asymmetry in each bin. We shall not attempt this here. We only wish to note that the 
third bin is automatically identified once the magnitude of S32 is determined. It may also 
be possible to combine the results of several bins (with appropriate sign) to improve the 
statistics. 

Note that with an energy cut E > 4 GeV, the events are roughly distributed in the 
proportion 2.5 : 2 : 1 from deep inelastic scattering (DIS), resonance (dominated by 1-pion 
processes) (RS), and quasi-elastic (QE) charged current processes respectively. There are 
roughly 3000 events in 480 kton-years, of which about 35% are up-going ones. 

Finally, if a non-zero asymmetry is measured, this will yield both the mass hierarchy as 
well as a lower bound on 613 of at least 613 > 6° (sin^ 26'i3 > 0.04). While the amplitude of 
the asymmetry is proportional to sin ^13, limitations due to statistics make it unlikely that 
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the value of ^13 itself will be determined with any reasonable accuracy. One needs to address 
this issue elsewhere. 



B. Asymmetries with finite detector resolution 

Inclusion of finite detector resolution reduces the sensitivity, especially beyond the first 
oscillation minimum and maximum. A straightforward way to include such effects is to 
smear the observed L and E and observe the impact of this. To this end, we define Gaussian 
resolution functions for both L and E: 



Ri{E',E) = }- exp 

V Sttcti 

Ri{lJ , L) = — — exp 

V 27r(72 



^ ' ■ (29) 



2al 



-{L-L 



l\2' 



2al 



Hence, the event rate now includes the probability that a neutrino of any possible L' and E' 
is detected in the detector with path-length L and energy E. We have 

D f f dE f'^ d^TV^ 



bin 



- ij'^L^l iVUV,L)^^ . (30) 



where"'' x' = L' / E'. 

We re-evaluate the event rates and the up/down asymmetries using this equation, with 
(Ti = 0.15E' and (T2 = 0.15L'. These are reahstic widths obtained by a GEANT analysis of 
atmospheric neutrino events by both the monolith and the ical/ino collaborations; this 
gives a typical FWHM for their ratio (L/E) to be about ~ OAL/E. 

The up/down events ratio (for the case of the direct hierarchy) is shown on the right 
side of Fig. 10 as a function of L/E for ^32 = 2 x 10~^ eV^. The results without including 
resolution functions is shown in the left-hand panel, for comparison. As before, a cut on the 
neutrino energy of > 4 GeV is apphed. With the inclusion of the resolution functions, it 
is seen that the amplitude of the oscillation is greatly damped. Furthermore, the neutrino 
rates ratio (shown as thick blue lines) is damped much more than that for the anti-neutrinos 
(the converse will be true for the inverted hierarchy). Curves correspond to 6*13 = 5, 11°. 
Clear oscillations are visible only around the first minimum and maximum. 

The resulting asymmetry, that is, the difference of the neutrino and anti-neutrino events 
ratios. An, is plotted in Fig. 11. Again, the panels on the right correspond to the rates 
calculation including resolution, while those on the left correspond to rates without including 
resolution effects. There is virtually no difference for small L/E. In general, however, the 
amplitude of the asymmetry decreases although it is still largest in the second envelope 
(corresponding to the third bin in the earlier discussions). 



The combination of Gaussian resolution functions for L and E integrates to a Lorentzian in 
their ratio provided the integration over all variables is unrestricted. We need to specify individual 
resolution functions here because of the cut on E. 
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It is seen that the asymmetry is larger with a cut of £"111111 = 4 GeV than with a cut of 
Erain = 2 GcV whilc the number of events is smaller by roughly half in the case of the larger 
energy cut. In particular, by lowering the cut to -Emin = 2 GeV, not only does the sample 
size effectively double, the contribution from various processes changes to the proportion 
1 : 1.5 : 1 from 2.5 : 2 : 1 for DIS, RS and QE processes. 

Thus there are two conflicting requirements: to enhance the size of the asymmetry, we 
need a larger value of -Kmiii) but to improve statistics, we need a larger sample size. The final 
choice of -Emin should optimise these two requirements. 

We note in passing that, for -Emin = 4 GeV, there is a fairly large asymmetry for large 
L/E as well; the statistical significance in this bin is however small. In both cases, the 
asymmetry in the penultimate envelope is almost vanishingly small; the oscillation length in 
this case is also very small. 

In short, it appears that a finite resolution function causes smearing of the data across 
bins in such a way that the magnitude of the asymmetry becomes significantly smaller, and 
even vanishes beyond the third envelope/bin. We can improve the result by summing the 
asymmetries over several envelopes. We define 

n 

K = E(-i)'^^" > (31) 

i=l 

where H corresponds to a given hierarchy, direct, or inverted, H = D.I. Typically, ~ 
—A^j^ in a bin, and therefore the sum in eq. 31 is roughly equal and opposite for the two 
hierarchies. Furthermore, the statistics are not very different with either hierarchy. The 
significance of the result is thus estimated by comparing the difference 

AAn ^A^-K (32) 

with the error a corresponding to a given exposure. In fact, the ratio A^„/(T indicates the 
probability of the observed asymmetry being assigned to the correct hierarchy. 

The error a arises primarily from statistics. This is because the asymmetry is defined 
as a difference of two ratios. Systematic errors (due to uncertainties both in fluxes and 
cross-sections) in the ratio are therefore very small and can be ignored; this was pointed out 
very early on by the monolith collaboration [10]. 

We list the values of AAn and the corresponding statistical error an (we have chosen 
the larger of the errors for the direct and inverted hierarchy) in Tables I, II, III for different 
resolutions in E and L of 15%, 10%, and 5% respectively. In each table, results for both n = 2 
and n — 3 (that is, a sum over two or three envelopes), corresponding to 1632] = 2.0 x 10~^ 
eV^ and ^13 = 7, 9, 11, 13° are shown for different exposures of 480, 800 and 1120 kton-years 
respectively. The resulting confidence level of the measurement is also indicated in the tables. 

It is clear that the asymmetry and hence A An increases with ^13, independent of exposure. 
The effect of increasing the exposiuc is to improve the errors, without changing the central 
values of the asymmetry. This can be seen by comparing results for the same 612 in each 
table. Since the minimum exposure of 480 kton-years is already large, and statistical errors 
improve only as V-^dato) it will be difficult to greatly enhance the significance of the result 
by going to larger and larger exposures. 
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The effect of improving tlie resolution function (obtained by comparing tlie same row in 
tlie tables I, II and III is two-fold. For a given exposure, improving the resolution function 
(by which we mean the improving the Gaussian widths of the resolutions in E and L as 
defined in eq. 30 above) increases the central value of the asymmetry in each case. It also 
results in marginally improved errors for n — 3, that is, when summing over all the three 
significant bins in the L/E distribution, which further enhances the significance of the result. 
For example, the improvement upon increasing the exposure from 480 to 1120 kton- years at 
15% resolution is similar to that obtained by improving the resolution, at 4.80 kton-years, 
from 15% to 10%. 

Note also that the significance of the results for AA2 is higher than for for larger 
resolutions of 15%; this may imply that best results are obtained from single-bin information 
when the resolutions are poorer. Also, it turns out that the results are systematically better 
for a cut of Emin = 4 GeV rather than for a cut of 2 GeV. 

While resolutions with widths of 10% may not be hard to achieve, it is not clear whether it 
is possible to reach 5%. One way is to exclude the DIS events (perhaps by cutting out events 
with large hadronic activity). The resulting data set is smaller by about 45%; however, it 
may be possible to resolve the energy and direction of the neutrino much better since there 
are only quasi-elastic and resonance events left in the sample. This needs to be studied in 
more detail. 

In summary, it appears that, even for moderate to large exposures, it may be difficult 
to achieve effects larger than 2a or so, unless of course Nature is kind and prefers large ^13. 
For 010, > 9°, a significance of 99.3% (nearly 3o") or more is obtained with an exposure of 800 
(1120) kton-years and resolutions in both energy and angle of 5% (10%). It is clear that a 
prior knowledge of ^13 from some other experiment would greatly facilitate the extraction of 
the mass hierarchy since the magnitude of the expected asymmetry would then be known. 

In short, the major features of the analysis without including finite detector resolutions 
appear to hold, although with much reduced significance. A prima facie case has been 
made for determining mass hierarchy through observation of atmospheric neutrinos with 
ICAL detectors; however, the effect is small and is difficult to measure. Indeed, it is clear 
that the asymmetry is very sensitive to smearing in both L and E. Our study indicates 
that an improvement in the detector resolution has a stronger impact on the results than a 
mere enhancement in size or the number of years of data-accumulation. A detailed study 
involving actual reconstruction of muon and hadron energies and muon direction is needed 
to determine the ultimate sensitivity of such a detector to the issue of hierarchy. 

C. The Sum 

The results from the sum of neutrino and anti-neutrino events <Sjv are much less sensitive 

to matter effects, as expected. Results (with same cuts as above) are shown in Figs. 12, 13 
and 14 for \632\ = 1, 2, 3 x 10^"^ eV^ respectively. For large values of ^13 > 7°, the amplitude 
of the sum averages to 1/2 much faster for the direct hierarchy. The numerical study will 
need to be carefully done to estabhsh the significance of this result. 
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V. DISCUSSION 



We have discussed the possibihty of obtaining information on the mixing angle 613 and 
the sign of using a magnetised iron calorimeter collecting data on atmospheric muon 
neutrinos, capable of charge identification. It appears that such a detector will be sensitive 
to the neutrino mass hierarchy only if 6*13 > 6° (sin^ 26'i3 > 0.04). 

Wc have defined a difference asymmetry in eq. 22 as the difference between the up/down 
rates of neutrinos and anti-neutrinos. By taking the up/down ratio, the systematic errors are 
greatly reduced. In particular, the uncertainty in the overall flux normalisation is removed. 
Such a ratio also minimises the dependence on the cross-section, which is substantially differ- 
ent for neutrinos and anti-neutrinos in the GcV energy range of interest. As a consequence, 
the measurable rate asymmetry is related to a theoretical flux-weighted probability (differ- 
ence) asymmetry given in eq. 24. Examination of the theoretical asymmetry immediately 
suggests ways of binning the data in order to maximise the asymmetry in each bin (in L/E). 
It turns out that the sign of the asymmetry is sensitive to the neutrino mass hierarchy (that 
is, the sign of ^32 = ml — mf) while its magnitude is sensitive to the (13) mixing angle ^13. 

We summarise our conclusions below. 

• The limits on 1^32 1 and sin 2^23 niay be reliably obtained using the location of the flrst 
minimum and the amplitude of flrst oscillation of a plot of the ratio of up/down events 
as a function of L/E. While no detailed discussion on this issue has been presented 
here, it has been well-estabhshed elsewhere [10,12]. 

The detector mass-exposure required for this purpose is about 150 kton-years. While 
charge identiflcation is not crucial for this step, presence of a magnetic fleld helps in 
obtaining better energy resolution. This step however is not dependent on the value of 
^13 or the sign of ^32. 

• If 9is is larger than 6°, then the pattern of subsequent oscillations after the flrst dip 
can give deflnite indication about the hierarchy through the difference asymmetry as 
discussed above. It is possible to maximise the sensitivity to this asymmetry in flxed 
L/E bins (improve statistical signiflcance) by applying an energy cut, E^i^ > 4 GeV. 
An optimum 1^32 1 dependent bin- width can be determined that is matter independent. 
This is easily accomplished as \532\ can be well determined independently as discussed 
above. 

For direct hierarchy the matter effects dominate through the neutrinos which have 
a higher cross section and hence give more pronounced matter effects. If the matter 
effects are small even when ^13 is large, then it is a clear indication of inverted hierarchy 
since the dominant neutrino events will have less impact due to matter. 

• Inclusion of flnite detector resolution reduces the sensitivity, especially beyond the flrst 
oscillation minimum and maximum in L/E. A combination of enhanced exposure and 
improved resolution will still enable a measurement of the mass hierarchy, provided ^13 
is not too small, ^13 > 6°. The best scenario is when ^13 is already established from 
other experiments to be in the favourable range. Of course, in the absence of such a 
measurement, non-observation of the asymmetry can put a lower bound on ^13 while 
leaving open the issue of the hierarchy. 



18 



• It is therefore important to maximise the sensitivity of the detector to the range of 
L/E where the matter effects are significant. It may be interesting to consider a 
non-horizontal detector geometry in order to maximise the sensitivity to the relevant 
L/E bins; typically, these correspond to an average path-length L around 7000 km. If 
neutrinos with such path-lengths are to be transverse to the iron plates, a geometry 
with plates tilted at about 33° to the horizontal may be preferred. This needs more 
careful study. 

While many experiments (present and proposed) will be sensitive to the (13) mixing 
angle ^13, it appears that charge identification may be essential for settling the issue of the 
mass hierarchy. In this context we note that reactor neutrino experiments arc not sensitive 
to matter effects and will be complementary to an ICAL detector. Short-baseline accelerator 
experiments may also not be as sensitive to matter effects. Long-basehne experiments with 
neutrino factory beams can surely settle this issue, and are sensitive to much smaller values 
of ^13, but are expected to come on-line at much later times. 

Thus a study of atmospheric neutrinos with a magnetised iron calorimeter detector can, 
in principle, provide fundamental information on the parameters of the neutrino mixing 
matrix and the neutrino mass-squared differences; in particular, it may resolve the question 
of hierarchy in the neutrino masses. It is certain that large exposures (more than 500 kton- 
years) will be required to obtain a result with a significance of at least 90% CL. In addition, 
good resolution in both energy and angle discrimination is required. For larger 9i^, > 9°, a 
significance of 99.3% (nearly 3o") or more is obtained with an exposure of 800 (1120) kton- 
years and resolutions in both energy and angle of 5% (10%). Results for smaller 61^ remain 
at the (2(t) 95% CL even in the best-case scenario. Hence the practical issue of experimental 
feasibility seems much harder to resolve. 

Acknowledgements: We are grateful to Dave Casper for making the NUANCE software 
freely available, and answering a long list of questions on its use. We are grateful to the 
members of the INO group, and in particular to the members of the INO group at IMSc, K. 
Kar, H.S. Mani, G. Rajasekaran, and Abdul Salam for many discussions and encouragement. 
We also thank G. Rajasekaran for a careful and critical reading of the manuscript and many 
suggestions for its improvement. We thank Nita Sinha and Probir Roy for pointing out 
errors in some analytical formulae of Section IIB. 



19 



REFERENCES 

[1] S.N. Ahmed et al. (SNO Collaboration), nucl-ex/0309004; Q.R. Ahmad, et al.(SNO 

Collaboration), Phys. Rev. Lett. 89(2002)011302; Phys. Rev. Lett. 89 (2002) 0113010, 
Phys. Rev. Lett. 87 (2001) 071301; S. Fukuda, et al. (Super-Kamiokande Collaboration), 
Phys. Lett. B539 (2002) 179, Phys. Rev. Lett. 86 (2001) 5656; Phys. Rev. Lett. 86 (2001) 
5651-5655; hep-ex/0103032; Fukuda, Y. et al. (Super-Kamiokande Collaboration), Phys. 
Rev. Lett. 82 (1999) 2430; Phys. Rev. Lett. 82 (1999) 1810; hep-ex/9812009. Phys. Rev. 
Lett. 81 (1998) 1158; W. Hampel, et al. (GALLEX Collaboration), Phys. Lett. B447 
(1999) 127; J.N.Abdurashitov, et al. (SAGE Collaboration), Phys. Rev. Lett. 83 (1999) 
4686; K.S. Hirata, et al. (Kamiokande-11 Collaboration), Phys. Rev. D44 (1991) 2241; 
R. Davis, (Homestake) Prog. Part. Nucl. Phys. 32 (1994) 13-32. 

[2] Y.Fukuda, et al. (Super-Kamiokande Collaboration), Phys. Rev. Lett. 81 (1998) 1562; 

[3] K. Eguchi et al. (KaniLAND Collaboration), Phys. Rev. Lett. 90 (2003) 021802, hep- 
ex/0212021; ApoUonio, M. et al. (CHOOZ Collaboration), Eur. Phys. J. C27 (2003) 331, 
hep-ex/0301017. 

[4] Y. Ashie et al. (Super-Kamiokande Collaboration), hep-ex/0404034; M. Ishitsuka 
(Super-Kamiokande Collaboration), NOON2004, The 5th Workshop on "Neutrino Os- 
cillations and their Origin", 11-15 February 2004, Tokyo, Japan. 

[5] B. Pontecarvo, J. Exp. Theor. Phys. 33, 549 (1957); Sov. Phys. JETP, 6,429 (1958). 

[6] Z. Maki, M. Nakagawa and S. Sakata, Prog. Theor. Phys. 28,870 (1962) 

[7] M.H. Ahn, et al. (K2K Collaboration), Phys. Rev. Lett. 90 (2003) 041801, hep- 
ex/0212007. 

[8] T. Tabarelli de Fatis, arXiv: hep-ph/0202232. 
[9] Sergio Palomares-Ruiz and S. T. Petcov, arXiv: hep-ph/0406096 
[10] N.Y. Agafonova et al.. The MONOLITH proposal, LNGS P26/2000, CERN/SPSC 2000- 

031, August 2000 (http://castore.mib.infn.it/~monolith/proposal/). 
[11] A. Para, arXiv: hep-ph/0005012; C. Andreopoulos, P. Stamoulis and G. Tzanakos, 

NuMLNote-ATM_NU-990, UA/ Phys/ HEP/ 11-05-2003 (http://www-numi.fnal.gov/). 
[12] G. Rajasekaran, arXiv: hep-ph/0402246 (http://www.imsc.res.in/~ino). 
[13] NUANCE Neutrino Generator, D. Casper, Nucl. Phys. Proc. Suppl. 112 161 (2002) 

(http:/ /www. ps.uci.edu/ ~nuint/nuance/default.htm). 
[14] M. Honda, T. Kajita, K. Kasahara, S. Midorikawa, Phys. Rev. D 64 053011 (2001). 
[15] J.N. Bahcall, M.C. Gonzalez-Garcia, C. Pena-Garay, JHEP 0302 (2003) 009, hep- 

ph/0212147; M.C. Gonzalez-Garcia, preprint hep-ph/0211054, talk given at the 10th 

international conference on supersymmetry and unification of fundamental interactions, 

SUSY02, DESY, Hamburg, June 2002. 
[16] M.C. Gonzalez-Garcia and Y. Nir, CERN-TH/2002-021, arXiv: hep-ph/0202058. 
[17] V.D. Barger, S. Pakvasa, K. Whisnant, and R.J.N. Phillips, Phys. Rev. D 22 2718 

(1980). 

[18] A. M. Dziewonski and D.L. Anderson, Preliminary reference Earth model (PREM), 

Phys. Earth Plan. Int. 25, 297 (1981). 
[19] P. Huber and W. Winter, Phys. Rev. D 68 037301 (2003), arXiv: hep-ph/0301257. 
[20] P. Picchi and F. Pietropaolo, ICGF RAP. INT. 344/1997, Torino 1997 (CERN preprint 

SCAN-9710037). 



20 



TABLES 



Exposure (kton-year) 


AA2 (CL %) 


A^3 (CL %) 


480 
800 
1120 


^13 = 7° 
0.166 ±0.161 (1.0 cr, 68.3%) 
0.166 ±0.125 (1.3(7, 80.6%) 
0.166 ±0.105 (1.6 cj, 89.0%) 


0.167 ±0.230 (0.7(7, 51.6%) 
0.167 ±0.178 (0.9(7, 63.2%) 
0.167 ±0.151 (1.1(7, 72.9%) 


480 
800 
1120 


^13 = 9° 
0.257 ±0.159 (1.6 £7, 89.0%) 
0.257 ±0.123 (2.1a, 96.4%) 
0.257 ±0.104 (2.5(7, 98.8%) 


0.280 ±0.230 (1.2(7, 77.0%) 
0.280 ±0.178 (1.6(7, 89.0%) 
0.280 ±0.150 (1.9(7, 94.3%) 


480 
800 
1120 


On = 11° 

n 957 _|_ n 1 57 (2'\rT 97 9%") 
0.357 ± 0.122 (2.9(7, 99.6%) 
0.357 ±0.103 (3.5(7, 99.95%) 


n 41 5 _|_ n o-^n f1 8(7 92 8%"! 
0.415 ±0.178 (2.3(7, 97.9%) 
0.415 ±0.150 (2.8(7, 99.6%) 


480 

800 
1120 


013 = 13° 
0.459 ± 0.155 (3.0(7, 99.7%) 
0.459 ±0.120 (3.8(7, 99.98%) 
0.459 ±0.102 (4.5(7, 99.999%) 


0.560 ±0.230 (2.4(7, 99.4%) 
0.560 ±0.178 (3.1(7, 99.8%) 
0.560 ± 0.151 (3.7(7, 99.98%) 


TABLE I. Significance of the asymmetry for different exposures in kton-years for resolution 
widths (Ji,2 in E and L (see eq. 30) of 15%. Asymmetries are shown for typical values of 
di3 = 7, 9, 11, 13°; the CHOOZ bound limits this angle to 9i3 < 14.9°. 


Exposure (kton-year) 


A^2 (CL %) 


A^3 (CL %) 


480 
800 
1120 


013 = 7° 
0.185 ±0.167 (1.1(7, 72.9%) 
0.185 ±0.130 (1.4(7, 83.8%) 
0.185 ±0.110 (1.7(7, 91.1%) 


0.232 ±0.220 (1.1(7, 72.9%) 
0.232 ±0.170 (1.4(7, 83.8%) 
0.232 ±0.144 (1.6(7, 89.0%) 


480 
800 
1120 


013 = 9° 
0.287 ±0.165 (1.7(7, 91.1%) 
0.287 ± 0.128 (2.2(7, 97.2%) 
0.287 ±0.108 (2.7(7, 99.3%) 


0.384 ±0.220 (1.8(7, 92.8%) 
0.384 ±0.170 (2.3(7, 97.9%) 
0.384 ±0.144 (2.7(7, 99.3%) 


480 
800 
1120 


013 = 11° 
0.399 ±0.163 (2.4(7, 98.4%) 
0.399 ± 0.126 (3.2(7, 99.9%) 
0.399 ±0.107 (3.7(7, 99.98%) 


0.565 ±0.221 (2.6(7, 99.1%) 
0.565 ±0.171 (3.3(7, 99.9%) 
0.565 ±0.144 (3.9(7, 99.99%) 


480 

800 
1120 


013 = 13° 
0.515 ±0.161 (3.2(7, 99.9%) 
0.515 ±0.125 (4.1(7, 99.996%) 
0.515 ±0.102 (4.9(7, 99.9999%) 


0.758 ±0.223 (3.4(7, 99.9%) 

0.758 ±0.173 (4.4(7, 99.999%) 
0.758 ± 0.146 (5.2(7, 100%) 



TABLE II. Same as Table I for resolution widths C7i^2 in E and L (see eq. 30) of 10%. 
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Exposure (kton-year) 


AA2 (CL %) 


A^3 (CL %) 


480 
800 
1120 


^13 = 7° 
0.198 ±0.172 (1.1 c7, 72.9%) 
0.198 ±0.133 (1.5(7, 86.6%) 
0.198 ±0.112 (1.8(7, 92.8%) 


0.297 ±0.209 (1.4(7, 83.8%) 
0.297 ±0.162 (1.8(7, 92.8%) 
0.297 ±0.137 (2.2(7, 97.2%) 


480 
800 
1120 


013 = 9° 
0.308 ±0.170 (1.8(7, 92.8%) 
0.308 ± 0.132 (2.3(7, 97.9%) 
0.308 ±0.111 (2.8(7, 99.6%) 


0.489 ± 0.209 (2.3(7, 97.9%) 
0.489 ±0.162 (3.0(7, 99.7%)) 
0.489 ±0.137 (3.6(7, 99.97%) 


480 
800 
1120 


013 = 11° 
0.429 ± 0.168 (2.5(7, 98.8%) 
0.429 ± 0.130 (3.3(7, 99.9%) 
0.429 ±0.110 (3.9(7, 99.99%) 


0.717 ±0.213 (3.4(7, 99.9%) 
0.717 ±0.165 (4.3(7, 99.9983%) 
0.717 ±0.139 (5.1(7, 100%) 


480 
800 
1120 


013 = 13° 
0.554 ±0.166 (3.3(7, 99.9%) 
0.554 ±0.128 (4.3(7, 99.998%) 
0.554 ±0.109 (5.1(7, 100%) 


0.960 ±0.218 (4.4(7, 99.999%) 
0.960 ±0.169 (5.7(7, 100%) 
0.960 ±0.143 (6.7(7, 100%) 



TABLE III. Same as Table I for resolution widths ai^2 in E and L (see eq. 30) of 5%. 
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FIGURES 



1.2 
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L/E 



FIG. 1. Survival probability of muon neutrinos in vacuum with two and three neutrino mixing 
scenarios. We have used \632\ = 2 x 10^'^, 623 = 7r/4 in both. For the three generation probability 
we have used in addition ^13 = 9° and the best fit values of other parameters as given in the text. 
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FIG. 2. A schematic of the relation between up-coming and down-going neutrinos with 

{-K- 6). 
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FIG. 3. Up/down event ratios for shown as thick (blue) Hnes and z/^ as thin (red) Hnes as a 
function of L/E for E > A GeV. The three horizontal panels correspond to 6 = \632\ = 1, 2, 3 x 10~^ 
eV^. The vertical panels correspond to the direct (^32 > 0) and inverted {6^2 < 0) hierarchy for 
^13 = 5°. All other parameters are kept at their best fit values as given in the text. 
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FIG. 4. Same as in Fig.3 with 6*13 = 11°. 
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FIG. 5. The difference asymmetry (difference of up/down ratios of muon neutrinos and 
anti- neutrinos) as a function of L/E for E > A GeV. The three panels correspond to 
5 = |(^32| = 1,2,3 X 10~^ eV^. The thick (blue) curves correspond to the direct S32 > and 
the thin (red) curves correspond to the inverse < hierarchy. The innermost curve in each 
envelope corresponds to ^13 = 5° and the outermost corresponds to ^13 = 11° with 7° and 9° in 
between. The best-fit values of other parameters are given in the text. 
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FIG. 6. The total up/down ratio including both muon neutrinos and anti- neutrinos as a function 
of L/E ioi E > i GeV. The three panels correspond to 5 = |532| = 1, 2, 3 x lO'^ eV^. The best-fit 
values of other parameters are given in the text. The thick (blue) curves correspond to the direct 
{632 > 0) and the thin (red) curves correspond to the inverted (532 < 0) hierarchy. The curves 
correspond to ^13 = 5° and 11° as indicated. Curves for in-between values of ^13 will lie in-between. 
Matter effects are more pronounced in the direct hierarchy. 
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FIG. 7. The up/down Monte Carlo events asymmetry An integrated over L/E bins shown 
as a function of L/E. The dots and crosses correspond to the direct and inverted hierarchies 
respectively. Error bars shown are obtained from the number of events in each bin. The histograms 
correspond to the same asymmetry calculated from theory; see Fig. 5. Thick (blue) lines refer to 
the direct hierarchy and thin (red) ones to the inverted one. The effect of fluctuations is indicated 
by the shift in the central data values from the histogram. Results are shown for ^13 = 5, 7, 9, 11 
degrees for \S32\ = 1 x 10~^ eV^. For more details on the bins, see the text. 
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FIG. 8. The same as Fig. 7 for l^ss] = 2 x lO'^ eV^. 
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FIG. 9. The same as Fig. 7 for l^ss] = 3 x lO'^ eV^. 
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FIG. 10. A comparison of the up/down events ratio as a function of L/E (for the direct hierar- 
chy) with (right) and without (left) the inclusion of Gaussian resolution functions for | (532 1 = 2 x 10~^ 
eV^ and 6*13 = 5,11°. Results are very sensitive to the widths of the Gaussian used. The results 
presented correspond to a Gaussian width a = 15% for both L and E. The thick (blue) curves 
correspond to neutrinos and the thin (red) curves to the anti-neutrinos; note that the latter are 
barely discernible from each other. 
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FIG. 11. A comparison of the difference asymmetry as in Fig. 5 with (right) and without (left) 
inclusion of resolution functions for 6 = \632\ = 2 x 10~^ eV^ and ^13 = 5, 11°. The thick (blue) 
curves correspond to the direct and the thin (red) curves to the inverted hierarchy. Two cases, 
E > 4 and E > 2 GeV, are shown. It is clear that the higher energy cut better emphasises the 
asymmetry and hence the matter effects; however, the event rates are roughly halved in this case, 
for the same exposure times. 
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FIG. 12. The up/down ratio of the sum of neutrino and anti-neutrino events integrated over 
L/E bins shown with statistical errors as a function of L/E. Results are shown for ^13 = 5, 7, 9, 11 
degrees for | J32I = 1 x 10~^ eV^. For more details, see the caption of Fig. 7. 
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FIG. 13. The same as Fig. 12 for l^gs] = 2 x IQ-^ eV^. 
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FIG. 14. The same as Fig. 12 for l^gs] = 3 x IQ-^ eV^. 
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